Authenticating and authorizing users regarding physical goods

ABSTRACT

Briefly, a portable intelligent device is provided that has an audio input for receiving a voice input from a user and an event manager for detecting that an event has occurred. The intelligent device also stores a passcode and a voice-code indicative of the passcode that is unique to a particular user. The intelligent device presents the passcode to a user, for example, from a display on the device, or from smart phone or tablet wirelessly connected to the intelligent device. The user speaks the passcode into an input transducer (microphone) on the intelligent device, and a processor generates a voiceprint that reflects the spoken passcode. The processor then can use the stored voice-code and the generated voiceprint to determine if a specific user was speaking, and if the user spoke the correct passcode. In this way the intelligent device is able to authenticate or authorize a remote user simply by having the user anonymously speak a passcode into the intelligent device.

RELATED APPLICATIONS

This application claims priority to U.S. provisional patent applicationNo. 62/732,660, filed Sep. 18, 2018, and entitled “User Specific ActionsRegarding Physical Goods.” This application is related to U.S. Pat. No.10,152,905, entitled “Symbol Verification for an Intelligent LabelDevice” and to U.S. Pat. No. 10,078,977, entitled “Optically DeterminingMessages on a Display,” both of which are incorporated herein byreference. This application is also related to U.S. patent applicationSer. No. 15/228,270, entitled “Transaction Agents and Systems.” to U.S.patent application Ser. No. 15/668,482, entitled “Verifying MessagesProjected From an Intelligent Audible Device,” to U.S. patentapplication Ser. No. 15/602,885, entitled “Agents and Systems for RightsManagement,” to U.S. patent application Ser. No. 15/890,312, entitled“Polymorphic Electro-optic Displays,” to U.S. patent application Ser.No. 16/010,485, entitled “Intelligent Container System,” to U.S. patentapplication No. 62/760,534, entitled “Fail-safe Intelligent Label andContainer System,” and to U.S. patent application No. 62/890,479,entitled “Autonomous Multi-medium IoT Devices and Systems,” all of whichare incorporated herein by reference.

FIELD OF THE INVENTION

The field of the present invention is portable electronic devices havinga processor for determining what user is proximate the device, andauthenticating or authorizing that user.

BACKGROUND

In the world of IoT (Internet of Things) and smart devices, desirableoutcomes often depend on the participation of a specific user (or one ormore specific users, or a group of specific users) in the presence of,or proximate to, a specific “thing”. Of particular interest are thingsor “goods” that are transportable or mobile, or otherwise have multiplecustodians or users.

Determining, and appropriate to the use, authenticating or authorizing(granting permissions) that a specific user (“user”) in proximity of aparticular good is important in many applications and to their outcomes,e.g.,

-   -   Adapting instructions regarding a thing to a specific user, and        monitoring a user's compliance with them (e.g. patient        prescriptions and compliance with them)    -   Providing one-time or limited authorization to enter or exit a        secure facility, use an item, or access information or rights    -   Confirming delivery of a particular item to, and acceptance by,        a particular individual

In general, such applications involve one or more of the followingregarding a particular thing:

-   -   authenticating specific users (proximate the thing)    -   authorizing, affecting, or effectuating local actions or        transactions by specific users    -   associating specific users with local actions or remote        transactions

Of particular interest are user authentication or authorizationactivities that relate to locally determined conditions or circumstancesof a good, a user or their location. And further, that preferably arenot dependent on

-   -   A password/code or other information that needs to be        remembered, stored or retrieved by the user    -   A specific appliance (e.g., a registered mobile phone or        notebook or tablet)    -   Limitation to a particular location (e.g. due to ‘fixed’ user        communication device or IP address)    -   An always-on internet/remote connection    -   Foregoing privacy or security

Accordingly, disclosed herein are novel systems and methods forauthenticating and authorizing specific users proximate a good usingtheir vocal characteristics without comprising their privacy.

SUMMARY

A portable intelligent device is provided that has an audio input forreceiving a voice input from a user and an event manager for detectingthat an event has occurred. The intelligent device also stores apasscode and a voice-code indicative of the passcode that is unique to aparticular user. The intelligent device presents the passcode to a user,for example, from a display on the device, or from smart phone or tabletwirelessly connected to the intelligent device. The user speaks thepasscode into an input transducer (microphone) on the intelligentdevice, and a processor generates a voiceprint that reflects the spokenpasscode. The processor then can use the stored voice-code and thegenerated voiceprint to determine if a specific user was speaking, andif the user spoke the correct passcode. In this way the intelligentdevice is able to authenticate or authorize a remote user simply byhaving the user anonymously speak a passcode into the intelligentdevice.

In one example, the intelligent device is integrated into a prescriptionpill bottle. The bottle has been filled by the pharmacist for a specificpatient, and the pharmacist loaded the passcode, the patient'svoice-code and the patient's prescription into the intelligent devicebefore mailing the pill bottle to the patient. In response to a passcodepresented by the intelligent device (for example when it is time to openthe pill bottle and take their medication), the patient speaks thepasscode into the integrated microphone of the intelligent device. Theprocessor in the intelligent device then authenticates the patient byevaluating the spoken passcode with the stored voice-code. If theevaluation is successful, the pill bottle stores a record of the resultsand resets the display. If the evaluation isn't successful, the pillbottle may present a visible alert and wirelessly transmit an alarmmessage. In another example, the intelligent device may communicate backto the pharmacist that the intended patient has received the medication.Or if wrong user speaks the passcode, then the pill bottle may generatean alert or alarm and transmit a message to the pharmacist.

Advantageously, the intelligent device is able to confidently determineif a specific user was in the presence of a good, such as a pill bottle,and they were taking a correct action (e.g. opening or closing the pillbottle). Further, since the events are time-stamped, a complete historyof how the user interacted with the good may be constructed.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram representing a simple authentication andauthorization ecosystem

FIG. 2. is a block diagram showing generation of a native voice-code.

FIG. 3. is a block diagram showing generation of a synthetic voice-code.

FIG. 4A. is a block diagram representing an exemplary applicationcomprising an intelligent pill bottle and local evaluation of avoiceprint and a voice-code.

FIG. 4B. is a process block diagram representing an exemplaryapplication comprising an intelligent pill bottle and local evaluationof a voiceprint and a voice-code.

FIG. 5A. is a block diagram representing an exemplary applicationcomprising an intelligent pill bottle, a user communication device andlocal evaluation.

FIG. 5B. is a process block diagram representing an exemplaryapplication comprising an intelligent pill bottle, a user communicationdevice and local evaluation.

FIG. 6A is a block diagram representing an exemplary applicationcomprising an intelligent pill bottle, user communication device andremote evaluation.

FIG. 6B. is a process block diagram representing an exemplaryapplication comprising an intelligent pill bottle, user communicationdevice and remote evaluation.

FIG. 7 is a block diagram of an exemplary application comprising anintelligent pill bottle, ePRO user communication device and localevaluation.

FIG. 8 is a block diagram of an exemplary application comprising anintelligent pharmaceutical container system and remote evaluation.

DESCRIPTION

As the term is used herein “things” are physical items or “goods” (theterm used here forward) that are the focus of local user actions and/orremote transactions. Goods may be consumables, disposable or durable;raw materials, components, sub-assemblies or finished goods;single/multi-use; perishable or non-perishable; tools, items ofequipment, appliances, machines, dispensers (e.g. liquids, foods,tickets, products), vending machines, conditional use/access systems,(e.g. parking meters, rental scooters or bicycles) locks and locking(and unlocking) systems; vehicles (e.g., autonomous delivery vehicles ordrones) etc.

Intelligent devices (or the functions thereof) are attached or coupledto, embedded within, or otherwise bound to goods. Exemplary intelligentdevices include smart labels, tags, containers, packaging, stored valueand security cards, containers etc. Of particular interest areintelligent “hardware agents” and related systems as described in U.S.patent application Ser. No. 15/228,270 Transaction Agents and Systemsand Ser. No. 15/602,885 Agents and Systems for Rights Management. Unlessstated otherwise, the term intelligent device shall be understood toencompass hardware agents.

Intelligent devices generally comprise: a processor, a memory, aclock/timer, communication circuitry (typically wireless), a source ofpower (e.g. a battery or capacitor, or RF, optical, thermal ormechanical harvested energy), a user message medium (e.g. visual,acoustic or haptic) and an audio input transducer (e.g. a microphone)for capturing a user's voice. They may also include one or more sensors(e.g. temperature sensor, a shock sensor, a vibration sensor, a motionsensor, a pressure sensor, a strain sensor, a chemical sensor, aradiation sensor, a humidity sensor, an acoustic sensor, or a lightsensor) or actuators.

Intelligent devices comprise an event manager: functions or operationsperformed by the processor and circuitry coupled to clocks/timers,sensors, circuitry (e.g. location circuitry and communicationcircuitry), actuators etc. that collectively monitor and take actions inresponse to events during the lifecycle of the intelligent device/good.Exemplary events include local communications and interactions (e.g.mechanical, electrical, optical communications with users), actuation,elapsed time, changes in sensed or monitored environmental or internalconditions including mechanical action, detected sound or light, orheat, changes in location, tampering, malfunction, and loss of power.

Intelligent devices operate autonomously according to stored or embeddedinstructions (rules) that govern the device's operation. And further inthe case of hardware agents, enforce individual sets of instructions foreach of a plurality of stakeholders in the good or its outcomes,according to the respective rights of the stakeholders.

Intelligent devices may also include message verification circuitry suchas that disclosed in U.S. Pat. No. 10,152,905 Symbol Verification for anIntelligent Label Device, U.S. Pat. No. 10,078,977 Optically DeterminingMessages on a Display, U.S. Pat. No. 10,147,098 Symbol Verification foran intelligent Label Device and U.S. Pat. No. 10,168,298 ElectricallyDetermining Messages on an Electrophoretic Display, and U.S. patentapplication Ser. No. 15/668,482 Verifying Messages Projected From anIntelligent Audible Device and U.S. provisional patent application62/890,479 Autonomous Multi-medium IoT Devices and Systems. Messageverification circuitry can be used to increase confidence that passcodesand other information presented by user message mediums (e.g. thoseresponsive to evaluations as described below) are as intended, and ifnot, what messages were actually presented. Similarly, andcooperatively, verification circuitry can increase confidence inwireless communications (e.g. the broadcast of alerts and alarmsresponsive to evaluations).

In certain embodiments of the inventions described herein, a user mayuse a user communication device cooperatively with an intelligentdevice. A user communication device is typically configured to capture auser's voice (voice-data) and (1) accept input from a user (e.g. via akeyboard, keypad, touchscreen, microphone or camera), (2) wirelesslycommunicate with appropriately configured local intelligent devices(e.g. via RFID, Bluetooth/BLE, Wi-Fi, IR) and/or (3) communicate withremote entities via the internet/wide area networks. Exemplary usercommunication devices are mobile phones, tablets, notebooks as well asinternet connectable appliances. In certain applications a stationary,always connected/connectable variant of the user communication devicemay be advantageous. User communication devices generally do not need tobe associated with a specific individual.

Note that the configuration of the intelligent device may vary dependingon the role, if any, of a user communication device in a particularimplementation. For example, an intelligent device may not need amicrophone or a user message medium if they are provided by a usercommunication device.

The following is an example (FIG. 1) of a simplified user authenticationand authorization system 100 and method where:

-   -   A certificate authority 120 generates one or more passcodes and        voice-codes indicative of the passcodes    -   A distributor 110 (1) provisions an intelligent device 130 with        the passcodes and corresponding voice-codes, (2) binds the        intelligent device to a good (or the good's packaging or        container) and (3) distributes the good    -   intelligent device 130, responsive to an event, presents a        passcode 155 stored in its memory to a user 140 proximate the        good via a visual user message medium    -   The user 140 vocalizes the passcode 165    -   The intelligent device 130 (1) creates a voiceprint of the        user-vocalized passcode and (2) evaluates the voiceprint with        the voice-code to authenticate or authorize the user

A passcode is any code that may be visibly or audibly (or haptically)presented to a user by a user message medium (e.g. display or indicator,speaker (e.g., audio output transducer) or vibrator or surfacefluctuating device) of an intelligent device, or a user communicationdevice (e.g., a mobile phone or tablet) communicatively coupled to theintelligent device and configured to acquire the passcode and present itto the user via its own message mediums.

Passcodes may be fixed or variable, static or dynamic, public orprivate, single or multi-use, encrypted or unencrypted. Passcodes may beused in different combinations or series or randomly rotated. Passcodesmay be pre-printed (e.g. bar code or text string) on the intelligentdevice, packaging or good. One passcode may have multiple correspondingvoice-codes (e.g. to authenticate/authorize a group). A singlevoice-code may have multiple passcodes. A visually presented passcodemay be persistent, temporary (stable, volatile); switchable,self-switching, non-switchable, or a combination thereof. Passcodes maybe symbolic and associated with a plain text “key word” that can bespoken (e.g., an image of a “cat”, a shape “square” or color “blue”).Passcodes may also comprise a series of symbols or alphanumericcharacters presented concurrently or sequentially, and by differentcombinations of message mediums (e.g. audible or visual). An intelligentdevice may instruct a user as to what combination of symbols or theirattributes (shape, color, quantity, etc.) they should vocalize, and whatorder.

Passcodes are typically generated and managed by a certificateauthority. In some implementations stakeholders and users may howevergenerate their own passcodes. Passcodes are then loaded/stored(provisioned) into intelligent devices, typically by a 3rd party thatbinds the intelligent device, and the passcode(s) and correspondingvoice-code(s), to the good (e.g., a manufacturer, product packager,distributor, seller, transporter).

For the purposes of simplicity and clarity, the term certificateauthority shall refer to the entity that creates and manages thepasscodes (and voice-codes, user voice-profiles etc.). The entity thatgenerates the passcodes also typically manages the lifecycle of thepasscodes (expiration, renewal, use count etc.) and either independentlyor in cooperation with other parties, tracks user actions andtransactions based on the passcodes (authorization, denial of service oraccess etc.)

A stakeholder is any entity that has a stake in any outcome of a good orrelated systems, actions or transactions. A stakeholder may perform someor all of the functions of a certificate authority.

In healthcare/patientcare, exemplary stakeholders include healthcareprofessionals, professional offices/groups, clinics and hospitals (andlarger organizations comprising them), pharmacies, pharmaceuticalmanufacturers and distributors, insurance providers, employers,regulatory agencies and patients and their families, caregivers etc.Stakeholders may or may not be custodial and may or not be directlyinvolved in local actions or remote transactions involving or related tothe good, intelligent device or other stakeholders.

A voice-code is an analog or digital signature of a passcode vocalized(spoken) by a specific user, or, machine generated using a user'svoice-profile. FIG. 2 illustrates a voice-code generated from auser-vocalized passcode referred to herein as a native voice-code. FIG.3 illustrates a voice-code generated using a voice-profile, and referredto herein as a synthetic voice-code. Native voice-codes and syntheticvoice-codes are private and generated, stored and otherwise managed bysecure parties and systems (e.g. certificate authorities).

A voice-profile is a model of a user's vocal characteristics determinedfrom user voice-data that can be used to generate user-specificvoice-codes without the concurrent participation of the user. A user'svoice-profile can also be used to obtain the passcode underlying auser's voiceprint (described below).

A voiceprint is an analog or digital signature (analogous to afingerprint) of a user-vocalized passcode, where the passcode ispresented to the user in-situ by an intelligent device or a usercommunication device (described below). A voiceprint may be evaluated ina variety of ways to authenticate or authorize a specific user and takeactions accordingly.

One exemplary method of evaluating a voiceprint is pattern matching(e.g. comparing and evaluating a previously generated voice-code withthe locally generated voiceprint). This approach implicitly takes intoaccount differences in languages and regional dialects as well asdifferences in speech patterns and the vocal characteristics of theusers. The evaluation rules and systems may further employ systems andmethods for determining and evaluating the level of confidence in theresults of the voiceprint evaluation.

Evaluation at the site of the user and good can be performed in responseto a locally presented passcode, using a locally generated voiceprintand a previously generated, locally stored or remotely accessible,voice-code. In one preferred embodiment, the intelligent devicegenerates and stores the voiceprint, then performs the evaluation usinga voice-code previously stored in its memory. In other implementationsthe voiceprint is generated and stored in a user communication devicethat can access a previously generated voice-code stored in theintelligent device (or from the certificate authority e.g., using anidentifier from the intelligent device) and accordingly, perform theevaluation.

The location of the intelligent device, as self-determined withintegrated location circuitry (e.g. cellular, local area; location-basedservices, geo-fencing) can increase confidence that a user, is anintended user. For example, that the location of the intelligent deviceis consistent with the expected location of the user: e.g. the deliverylocation of a good to a specific buyer or consumer; the care facility ofa prescribed medication for a specific patient (e.g. their homeaddress), or the facility where permission to access is to be granted,or equipment is to be used.

In some implementations, the intelligent device will dynamicallygenerate the passcode (e.g. in response to life-cycle events).Evaluation of a related voiceprint (described below) requires havingaccess to the passcode as well as the user's voice-profile (describedbelow) and a way to associate them. Since the passcode may be public foronly a short period, and in an adequately private environment, it may besufficient to digitally encrypt the passcode (and voiceprint) forcommunication to the device or entity performing the evaluation, e.g., auser communication device or certificate authority respectively.

Evaluation at a location remote from the user and good can be performedby a remote evaluation system, e.g., that operated by a stakeholder or acertificate authority. In a typical embodiment, the user communicationdevice sends from the user's location, a voiceprint (or voice-data) andan “identifier” (ID) to the remote evaluation system. The remoteevaluation system uses the identifier to retrieve a previously generatedvoice-code that corresponds to the received voiceprint, then evaluatesthe voiceprint with the voice-code. Alternatively, the usercommunication device also sends the passcode corresponding to thevoiceprint from the user's location, to the remote evaluation system.The remote evaluation system uses the identifier to

retrieve the voice-profile corresponding to the voiceprint andcontemporaneously generate a synthetic voice-code using the passcode andvoice-profile, which it then uses to evaluate the voiceprint. In avariation of the latter embodiment, the passcode could alternatively beretrieved by the remote evaluation system from a database of passcodesusing the identifier. Note that multiple parties, e.g. a stakeholder andcertificate authority) could collaboratively performed the functionsjust described.

Typically, the identifier is generated by a stakeholder or a certificateauthority and accompanies the intelligent device or companion good tothe user's location. In the simplest implementation, the identifier isused to determine and retrieve the previously generated voice-code to beused in the evaluation. The identifier can be private or public (e.g.,visibly—including printed, acoustically or wirelessly presented).Importantly, an association with a user's identity (or alias) is notrequired.

A valuable aspect of the authentication and authorization systems andmethods described herein is that the user can remain anonymous (the userisn't identified or identifiable), incognito (an alias is used—typicallyfor single or limited use) or identifiable (the user can be identifiedbut their identity is not necessarily public), or identified as a memberof a group of authorized users.

Note here and elsewhere, the evaluation process requires appropriatecircuitry/software (“evaluation circuitry”) located where the evaluationis performed (e.g. in the intelligent device or user communicationdevice if the evaluation is local to the good and user).Circuitry/software for generating voiceprints (“voiceprint circuitry”)is similarly required.

The results of an evaluation (e.g. authentication a user present a good)may authorize or initiate actions by the intelligent device or dependingon the implementation, an application on a user communication device.Exemplary actions by the such devices in response to an authorized uservoiceprint include enabling access to stored information, building ahistory of user specific interactions, sensing conditions, presentingmessages (e.g. access codes, alerts/alarms, financial or othertransactional information, enabling access to functions of, or operationof, a good etc.

The results of evaluations may also affect specific users' behaviors,e.g. taking actions or effectuating transactions regarding goods. Forexample, in response to previously described actions of an intelligentdevice. More expansively, the results of an evaluation that successfullyauthenticates specific user proximate a good, and at a particular momentin time, can be used to improve user compliance (see below) and optimizeuser behaviors that affect the outcomes of goods (and correspondingtransactions). Knowledge of a specific user's presence with a good at aparticular moment in, or period of, time can be advantageously combinedwith knowledge of the coincident actions taken by the intelligentdevice, e.g. visual, audible or wireless messaging presented the userand the corresponding actions taken (or not taken) by the specific userin response. And further, the results of an evaluation can be used toauthorize a specific user—grant them permissions, without having toidentify them.

The user's actions (or lack thereof) may be determined by an intelligentdevice (or user communication device). They may also be determined bycorrelating data received from an intelligent device and user-initiatedtransaction data from external system. An example of the latter being adecrement to an inventory system of a replacement good when the currentone is no longer fit-for use. Another example is a caregiver reporting(or not reporting) a message presented by an intelligent containercontaining opioids or other dangerous drugs that was generated becausethe container had been opened without an authorized user being present.

User compliance with the above processes can be optimized via variousperformance or outcome-based incentives (or disincentives). Patiencecompliance in healthcare for example, can be improved by offeringincentives (or disincentives) for acting appropriately to the messagingpresented by the bottle containing their drugs of when, and when not, totake them. And further, encouraging their proper storage/safety—e.g., tomake sure that any access by anyone other than the authorized patient isacted upon. Exemplary performance/outcome-based incentives include:

-   -   Waiving of fees such as insurance copays, reducing insurance        deductibles    -   Discounts on future prescription purchases    -   Automatic reordering of prescriptions (or rejection of refill        requests without consultation)    -   Multiple prescription dosage management at a discount

In one exemplary process 200 for generating a native voice-code (FIG. 2)a user using an audio enabled internet connected computing device (e.g.computer, notebook, tablet or mobile phone), establishes a securecommunication 203 with a certificate authority. The certificateauthority generates passcodes 205 and sends them to the user. The uservocalizes the passcodes (creating voice-data) 207 which are sent to acertificate authority. The certificate authority receives the vocalizedpasscodes and creates and stores corresponding native voice-codes 209.

In an exemplary process 300 for generating a synthetic voice-code (FIG.3), a user using an audio enabled internet connected computing deviceestablishes a secure communication with a certificate authority 308. Inresponse to the certificate authority, the user generates voice-data 312(e.g. the user vocalizes sample words and phrases prompted by thecertificate authority) that is captured and used by the certificateauthority to create a user voice-profile 314. The voice-profile can belater used to generate synthetic voice-codes based on then or later, thecertificate authority generates passcodes 316 that it uses to generatesynthetic voice-codes 318 when needed.

Advantageously such processes benefit from AI and in particulargenerative adversarial networks (GANs). In another process, voice-codescan be created simply by recording, and advantageouslyrefining/processing, and storing the user's vocalization of specificpasscodes. Various adaptive noise filtering/cancelling schemes can beused to compensate for variations in the acoustics of the user'senvironment and system used to create the voice-data.

Voice-codes and voice-profiles can be generated remotely via a secureprocess like the authorization of credit cards via internet or voiceactivation. Note that the process described above depends on anappropriate for the use, level of confidence that the person generatingthe voice-data is correctly identified and authorized. In other words,that person generating the voice-data is authenticated. Variousprocesses may be used for this purpose including two-factorauthentication, pre-defined voiceprints, or PINs.

To achieve a high level of confidence in the system, and higher qualityvoice-data, similar processes for generating voice-profiles andvoice-codes can be performed in a trusted facility of a stakeholder or acertificate authority (e.g. a doctor's office, pharmacy, retail clinic,hospital, or employer's human resource office; the office of a bank orother financial institution, or government office).

The stakeholder or certificate authority distributes passcodes, andvoice-codes according to the implementation, to the relevant elements ofthe ecosystem, e.g. for loading into intelligent devices orcommunication to appropriate user communication devices or authorizedstakeholders).

A certificate authority may provide services on behalf of multiplestakeholders (e.g. a patient, the patient's doctor, hospital, pharmacy,insurance company and regulatory agencies), thereby simplifyingadministration, maintaining a higher level of trust. For example, acertificate authority may perform some or all of the processes forcreating, storing, distributing and otherwise managing passcodes,voice-codes, voice-profiles, and evaluations etc. on behalf of thestakeholders.

A passcode may be presented to a user(s) for generating a voiceprint forlimited duration, number of times or time periods, or for specific timesor locations or other factors to encourage compliance and discouragemisuse and tampering. For example, a passcode may only be presentedconsistent with a patient prescription loaded in an intelligent pillbottle/container such as that described in U.S. patent application Ser.No. 16/010,485, An Intelligent Container System. Advantageously, theevaluation may be limited to a number of attempts or require secondaryactions.

A single intelligent device/good may have multiple authorized users andaccordingly multiple passcodes, multiple authorized voice-codes perpasscode or various combinations thereof. For example, an intelligentpill bottle might support an authorized patient, a family member, and/orhealthcare provider. A single intelligent device/good may also havedifferent instructions/rules conditioned on the determination ofspecific user or users.

As with previously described processes for generating voice-data,voice-codes and voice-profiles, adaptive noise cancellation or othersignal processing techniques may be used to improve performance whencreating voiceprints (e.g. to compensate for ambient sounds, orcharacteristics of the devices used, e.g. microphones/speakers).

Advantageously, acoustic watermarks and steganographic marks can beemployed to increase confidence in the determination of a specificuser's proximity to a good, and local actions and global actions thatdepend on them. For example, during the creation and evaluation of avoiceprint, or linked to specific, appropriate enabled hardware/devices(intelligent devices, mobile phones with specific/appropriatelyconfigured software/vocoders). Of particular interest are machinediscernible, inaudible (to humans) acoustic marks. And further, thoseoptimized for particular appliances/vocoders (e.g. mobile phones).

In one application of the systems and methods described herein, avoiceprint can be used to screen a user for being ‘fit-to-consume’ or“fit-to-use”. For example, is a patient awake and cognizant of thedecision/action to be taken?

In another application of the systems and methods described herein, auser's voice can be used to mark or otherwise secure thecollection/transmission of data/information to a ‘trusted’ stakeholder.This approach has the benefits of being device agnostic and the userdoesn't need to remember, look-up or safely store a security code. Andimportantly, it gives the user control over the securitization anddownstream use of their personal information.

For security and privacy reasons, it is preferable that at least aportion of an intelligent device's memory cannot be accessed externally(e.g. the portion that stores voice-codes or voice-profiles).

Also, for security and privacy reasons, it may be preferable that aftertheir useful life, or in the event of failed evaluations or tampering,that some or all of stored voiceprints, voice-codes and especiallyvoice-profiles, are self-erased (forgotten) by the intelligent device.

EXEMPLARY APPLICATIONS

The following examples illustrate the application of user authenticationand authorization systems and methods regarding a good. It is to beunderstood that only the salient elements are described and that manyother variations are possible consistent with the inventions describedherein. Each of the following examples are healthcare applications,however it should be understood that the inventions described areapplicable to a wide range of other uses.

Except as noted, central to each of the following example applicationsis an intelligent pill bottle (intelligent device) comprising:

-   -   A processor, a memory, a clock/timer and a battery    -   BLE, (or other local area wireless communication circuitry    -   Sensors/circuits for determining whether the pill bottle is open        or closed    -   A visual messaging medium(s)    -   A microphone for capturing the user-vocalized passcodes    -   Event and evaluation circuitry    -   A passcode and a voice-code stored in the memory

In the following examples, only one passcode and one voice-code areillustrated. In practice a single passcode could be reused, however amore secure option is to use multiple passcodes and voice-codes for thesame user (e.g. a patient in the following examples). Depending thecondition of the user (patient) and the presence of a trusted at-homecare provider, it may be advantageous to have one set of passcodes andtwo sets of corresponding voice-codes. In that way either the patient orthe care provider can be authenticated or authorized as appropriate tothe current circumstances. Note that a patient identifier is notrequire.

Exemplary Healthcare Application 1—Intelligent Pill Bottle and LocalEvaluation

The goals of this application are to (1) encourage compliance (takemedicines as prescribed), and (2) to discourage accidental,unintentional or intentional misuse. The solution is to determine if thepatient is in the immediate presence of their medication (an intelligentpill bottle when it was prescribed to be taken and took an actionindicative of compliance, e.g. the patient opened the pill bottle whenvisibly prompted to do so. FIGS. 4A and 4B illustrate an exemplaryhealthcare application 400, 450. In summary:

-   -   A pharmacy (stakeholder) 405        -   Obtains a passcode and corresponding patient voice-code from            a certificate authority 410 (using an anonymous ID)            associated with the patient 455        -   Wirelessly provisions the intelligent pill bottle 415 with            the patient's prescription (prescribed actions and timing            thereof) and the passcode and voice-code 460, and sets the            clock/timer    -   The patient 420 or a caregiver picks-up the intelligent pill        bottle and takes it home    -   When, according to the instructions operating in the intelligent        pill bottle (the prescription), it's time for the patient to        take their medication, the patient is visibly prompted (not        shown) via the intelligent pill bottle's visual messaging medium        465        -   When/if the pill bottle 415 is opened, the stored passcode            is visually presented 430 on its integrated visual message            medium 470        -   The patient 420 speaks (vocalizes) the passcode 435 to the            microphone in the intelligent pill bottle 475    -   The intelligent pill bottle generates and stores a voiceprint of        the user-vocalized passcode 480, and evaluates the voiceprint        with the voice-code stored in memory 485        -   If the voiceprint matches the voice-code (indicating that            the patient is present and complying with the prescription)            the intelligent pill bottle, erases the visual passcode from            the display        -   If the voiceprint does not match the voice-code, and            -   It's a near miss (likely to be the patient's voice, but                the evaluation isn't definitive), the agent generates a                visual prompt for the patient to try again            -   It's a clear miss (not likely to be the patient's voice)                the agent generates a visual alert and a wireless alarm                indicating use by someone other than the patient    -   If a valid voiceprint is not generated (or no vocalized pattern        is presented) within X minutes of the pill bottle having been        opened (indicating the possibility of unauthorized use), the        intelligent pill bottle generates visual alert and a wireless        alarm    -   If the intelligent pill bottle is not opened during the        prescribed time period, it generates a visual and optionally        wireless alert (e.g., to notify a caregiver)    -   When a valid voiceprint is generated, or if no valid voiceprint        is generated within X minutes (or Y attempts) of the passcode        being presented, the passcode is erased from the visual        messaging medium.

In all cases, the intelligent pill bottle maintains a secure log of allevents (actions, prompts, alerts, alarms etc.) and the times of them,for later review.

Preferably the visual messaging mediums described in these examples, andelsewhere herein are capable of presenting polymodal visible information(e.g. a polymodal display apparatus or a polymorphic display asdescribed in U.S. patent application Ser. No. 15/890,312 PolymorphicElectro-optic Displays). The intelligent pill bottles described hereinare related to intelligent container apparatuses, systems and methodsdescribed in U.S. patent application Ser. No. 16/010,485 IntelligentContainer System and U.S. patent application 62,760,534 Fail-safeIntelligent Label and Container Systems.

Exemplary Healthcare Application 2—Intelligent Pill Bottle, UserCommunication Device and Local Evaluation

This healthcare exemplary application 500, 550 (FIGS. 5A and 5B)illustrates how functions can be distributed between an intelligentdevice (the intelligent pill bottle of this example) and a usercommunication device communicatively coupled to the intelligent device.The intelligent pill bottle 515 in this example, is similar to that inthe first example (FIG. 4A), however it is not configured with amicrophone, voiceprint circuitry or a visible messaging medium forpresenting passcodes.

As in the first example, a pharmacy (stakeholder) 555 receives apasscode and voice-code from a certificate authority, and provisions theintelligent pill bottle with the passcode, voice-code and the patient'sprescription 560. The intelligent pill bottle 515 according to theprovisioned prescription (instructions) manages timing, duration andsequencing steps in the authentication processes.

The user communication device 530 (a tablet in this example), isconfigured with an appropriate app (or access to a webservice) visualmessage medium (display) for presenting passcodes, an acoustic messagemedium (speaker), microphone, input interface (acoustic ortactile—keypad, keyboard, touchscreen etc.) and voiceprint circuitry.The wireless communication circuitry in both the intelligent pill bottleand the user communication device is BLE (Bluetooth Low Energy).

As with the first example, when, according to the instructions operatingin the intelligent pill bottle 515 (the prescription), it's time for thepatient 520 to take their medication, the patient is prompted via theintelligent pill bottle's messaging medium (not shown) 565. Whenintelligent pill bottle is opened:

-   -   The tablet (user communication device) 530:        -   Wirelessly acquires the passcode 541 from the intelligent            pill bottle 515 and visually presents (and subsequently            erases) the passcode 543 for the patient 520 to vocalize 573        -   “Listens” to the patient vocalize (speak) the displayed            passcode 545, 577        -   Generates a voiceprint corresponding to the vocalized            passcode, and transmits it 547 to the intelligent pill            bottle 583    -   The intelligent pill bottle 515:        -   Wirelessly receives and stores the voiceprint transmitted by            the tablet        -   Evaluates the voiceprint against the voice-code stored in            its memory and stores the results of the evaluation 585        -   Generates and transmits alerts, alarms or messages 549            appropriate to the results of the evaluation to the tablet            530 (and/or another monitoring device or a remote entity not            shown).

Note that a variety of combinations and sequences of steps can beemployed in cooperation with the stored instructions (prescription) andoperation of an intelligent device (intelligent pill bottle). Forexample, the user could open the intelligent pill bottle, take theirmedication and close the pill bottle, and the authentication processcontinue without interrupting the desired patient behavior. In otherwords, the user communication device could be synchronized such that theauthentication process was implemented immediately prior to, during, orimmediately after the patient takes their medicine.

As appropriate to the application, the intelligent device and usercommunication device, together or individually, maintain logs of allevents (actions, prompts, alerts, alarms etc.) and their respectivetimes for later, secure access. The user communication device can alsobe used in cooperation with an intelligent device (and optionally alocal or remote wirelessly connected third party) to implementtwo-factor authentication processes.

Exemplary Healthcare Application 3—Intelligent Pill Bottle, UserCommunication Device and Remote Evaluation

This exemplary healthcare application 600, 660 (FIGS. 6A and 6B)illustrates how functions can be distributed between an intelligentdevice (the intelligent pill bottle of this example) and acommunicatively coupled user communication device (in this example aninternet connected, BLE and acoustically enabled appliance analogous toan Amazon Alexa or Apple Home Pod ‘smart speaker’), and a remotestakeholder or certificate authority (or other parties). In thisexample, the intelligent pill bottle, is configured with BLEcommunications, an ID (e.g. a prescription number printed on a label ofthe intelligent pill bottle), and visual message mediums for presentingpasscodes alerts/alarms and other messages. Unlike the previousexamples, the intelligent pill bottle in this example is not configuredwith evaluation circuitry; the evaluation is performed by a remotestakeholder.

As in the previous examples, a pharmacy (stakeholder) receives apasscode and voice-code from a certificate authority 655, and provisionsthe intelligent pill bottle with the passcode, voice-code and thepatient's prescription 660. The intelligent pill bottle 615 according tothe provisioned prescription (instructions) manages timing, duration andsequencing steps in the authentication processes.

According to the prescription provisioned in the intelligent pillbottle, a visual prompt is generated when it's time for the patient totake a pill 665. And as described above, according to the instructionsset in the intelligent pill bottle, the intelligent pill bottle presentsa visual passcode 633, 668. When the intelligent pill bottle 615 isopened:

-   -   The patient 620 “wakes up” 635 the user communication device        630, 672, enabling the appropriate voice-based application and        processes (i.e. Alexa skill). The voice-based application        prompts the patient to “speak” the passcode and ID 637 to the        user communication device 630, 676. The user communication        device then transmits the vocalized passcode and ID 643 to the        remote stakeholder or certificate authority 610.    -   The remote stakeholder/certificate authority 610        -   Receives 643 and converts the ID into text (speech-to-text)            or other machine processible code, and the vocalized            passcode into a voiceprint 682        -   Uses the text ID to determine (look-up) the appropriate            voice-code 682, and        -   Evaluates the voiceprint with the voice-code 685, and            according to the results of the evaluation executes one or            more transactions e.g., transmit instructions/data 645 to            the user communication device 630 or updates patient            records, flags abuse, low or high levels of compliance,            risky behavior, notifies other stakeholders (e.g. doctors,            hospitals, pharmacies)

Exemplary Healthcare Application 4—Intelligent Pill Bottle, ePRO UserCommunication Device and Local Evaluation

This example illustrates a system 700 (FIG. 7) comprising a simplifiedintelligent device and user communication device, where the usercommunication device performs the evaluation locally. The specificsystem illustrated is characteristic of ePRO (electronicpatient-reported outcomes) systems used in healthcare, and in particularclinical trials, that rely on dedicated user communication devices.Authenticating the user proximate the medication as the intendedpatient, improves accountability and compliance with the ePRO system andthe prescribed trial protocol.

In this example:

-   -   The stakeholder is a clinical trial operator 705    -   The intelligent pill bottle (intelligent device) 715 is        configured with        -   A processor, a memory, a clock/timer and a battery        -   Local area wireless communication circuitry (compatible with            that supported by the user communication device)    -    The intelligent pill bottle is not configured with a user        message medium or evaluation circuitry    -   The user communication device 730 is an internet connectable        tablet (in this example, provided by the clinical trial operator        and configured to support ePRO) configured with local area        wireless compatible with the intelligent pill bottles, a visual        user message medium, microphone (and in this example a speaker)        and evaluation circuitry.

As illustrated in FIG. 7:

-   -   A certificate authority 710 generates a passcode and a        voice-code    -   The clinical trial operator 705        -   Acquires the passcode and voice-code from the certificate            authority        -   Encrypts the voice-code (and optionally the passcode)        -   Provisions the intelligent pill bottle 715 with a passcode            and encrypted voice-code, and patient's prescription 741        -   Provides the tablet 730 (provisions or sends to the tablet)            the key to decrypt the encrypted voice code 743    -   At the patient's home, the tablet 730 wirelessly connects to the        intelligent pill bottle 715 and retrieves the prescription,        passcode and encrypted voice-code 744    -   The tablet        -   Audibly prompts 746 the patient when it is time to take            their medication        -   Visually presents 747 the passcode when the pill bottle is            opened (detected by the intelligent pill bottle and            wirelessly communicated to the tablet)        -   Listens to the patient vocalize the passcode 748 and            generates a corresponding voiceprint        -   Decrypts the encrypted voice-code using the key stored in            its memory        -   Evaluates the voiceprint with the decrypted voice-code to            authenticate the user as the intended patient (and includes            updates the ePRO system accordingly)

The intelligent pill bottle in the above example includes an internalbattery. Alternatively, the intelligent pill bottle could be configuredto harvest RF energy to power the circuitry (e.g. in cooperation withthe RF communication circuity and RF signals transmitted by the tablet).In another variant, the certificate authority encrypts the voice-code(and optionally the PC) and provides the key to decrypt the encryptedvoice-code directly to the table. There are of course many othersecure-data handling variations.

Exemplary Healthcare Application 5—Intelligent Pharmaceutical Containerand Remote Evaluation

This example is of a system 800 comprising an intelligent container 810(intelligent device) similar to the intelligent pill bottle 415 inexample 1, however in this case, constructed to be reusable, and furtherto include wide-area communication circuitry (e.g. cellular) and supportfor geolocation services. The wide area network enables remoteevaluation (akin to example 3) and the geolocation services establishthe location of the intelligent container and proximity of theuser/patient. Unlike the intelligent pill bottle 415, the intelligentcontainer 810 does not however, need evaluation circuitry nor local areawireless communication circuitry. The latter however would beadvantageous for provisioning the intelligent container, although thatcould be accomplished with the wide area communication circuitry.

In this example the stakeholder is a healthcare service provider 805that provides in-home patient care, and the intelligent container 810is:

-   -   Constructed for shipping/storing preformulated/prefilled        syringes and to be reusable    -   Configured with a processor, a memory, a clock/timer and a        battery, a visual user message medium, a speaker, a microphone,        sensors/circuits for determining whether the intelligent        container is open or closed, and cellular communication        circuitry

As illustrated in FIG. 8:

-   -   The healthcare service provider 805        -   Generates a passcode and a set of voice-codes indicative of            the passcode and plurality of authorized care givers (e.g.            staff of the healthcare service provider), and a container            ID, and further associates the container ID with the            voice-codes of the set of voice-codes        -   Activates the cellular service, and geolocation service        -   Provisions the intelligent container with the passcode and            prescription (stores the voice-code in a database) 831        -   Ships the intelligent container to a patient's home    -   The patient receives the connected intelligent container    -   The intelligent container 810        -   Audibly prompts the user when it is time to inject the            pharmaceutical 833        -   Visually presents the passcode to the user when the            intelligent container is closed 835        -   Listens to the on-site user vocalize the passcode and            generates a corresponding voiceprint 837        -   Wirelessly transmits over the cellular network, the            voiceprint and container ID to the healthcare service            provider 839    -   The healthcare service provider 805        -   Uses a location service (not shown) to determine the            location of the intelligent container and its proximity to            the patient's home        -   Identifies the set of voice-codes using the container ID        -   Evaluates the voiceprint with the voice-codes in the set of            voice-codes, and the determined geolocation, and            authenticate the specific caregiver on-site and their            proximity to the intelligent container

Note the above example, could be easily extended to also authenticatethe patient, their presence and being awake using a seconduser-vocalized passcode (the same or different passcode than the onepresented to the caregiver) and corresponding voice-print, and secondvoice-code for the patient.

In the all the previous examples, and in general all of the inventionsdescribed herein, applications, various forms of data security can beemployed.

Variations

Applications such as those described above, may be designed for multipleauthorized users. For example, with voice-codes for family members andhealthcare providers in addition to those for the patient. With multiplevoice-codes for example, an alternate authorized user can substitute forthe primary one (e.g. a pre-authorized healthcare provider cansubstitute for the patient). Multiple voice-codes (and preferably incombination with multiple passcodes) can also be used in applicationsbenefiting from the presence of both or two different parties (e.g. botha patient and a caregiver) have to be authenticated within a shortperiod of time of each other.

Users (e.g. patients), upon whose participation, compliance and actionsdesired outcomes depend, can be incentivized according to actual theiractions or outcomes (performance or outcome-based systems) withfinancial payments, credits, rebates, waivers or refunds of deductibles,insurance premiums and the like.

Deviations from prescribed actions can be identified and associated withspecific users and corresponding actions (or lack thereof) or otherevents related to user (e.g. patient) behavior.

Note that a “prescription” is an example of a set of rules set in theintelligent device. In the case of a prescription, the events arerelated to time. More generally, those rules are typically conditionalon local events. The local events being related to changes in internalor external conditions to the intelligent device. E.g., environmentalconditions, elapsed time (or actual time), user actions (e.g. I/O),changes in location, custody etc.

Outcomes that benefit or depend on authenticating and authorizing users(and thus their proximity to a good) can be strengthened with the use ofvoiceprint activated electro-mechanical locking mechanism and touchsensors. It is more likely for example, that a patient/user was actuallypresent holding/opening/closing the pill bottle (and thus more likely tocomplying with the prescription) if the intelligent device/bottle wastouched consistent with the prescription (and correlated with successfulauthentication and/or authorization of the user). As noted beforeneither authentication nor authorization depends on user identifiabledata.

The authentication and authorization systems and methods describedherein may be strengthened with audible and inaudible acousticwatermarks generated and locally propagated concurrent with presentationof passcode.

Authentication and authorization systems and methods comprisingintelligent devices can be optimized/fit-for-purpose in a variety ofways. An intelligent device can evaluate voiceprints of what was spokenin context of the action taken. For example: evaluate a voiceprint as“left” instead of “lift” depending on the context (e.g. the good, theuser or location). An intelligent device can verify the proximity ofperson (audible input) relative to action taken (using for examplesensing such as Bluetooth proximity). An intelligent device candetermine the number of respondents relative to action taken. Forexample, a loading dock with multiple workers and the handling of a goodor delivery vehicle. An intelligent device can match conditions (time,temperature, motion, . . . ) to audible input(s) and action(s) taken.

The authentication or authorization systems and methods described hereincan be used to authorize access to or use of physical or digital assets,based on voice, proximity of a user to an intelligent device andassociate good, or a secure input (by an authenticated or authorizeduser). Other uses include providing/authorizing rights based onpredetermined series of audible inputs (from the same or a plurality ofdifferent users), e.g. security guard making his rounds or shipper(delivery) and customer (acceptance); limit rights by detecting attemptsat (or actual) unauthorized use/access by via voice input, or limitpresented audible or visual information based on voices detected in aroom.

Voice-codes, voiceprints or voice-profiles can be secured as any type ofdigital object. Among the different security methods and processes are:

-   -   Encrypting the digital objects; breaking them into pieces and        storing in a distributed system, then reassembling when/where        and by what device as needed    -   Converting the voiceprint data to a token or secure code that        references the voiceprint; adding a watermark to voiceprint.        This is an extension of audio watermarking which is used to        identify if part or all of a voiceprint (or other audio) has        been stolen or inappropriately reused.    -   Adding a time-bomb to voiceprint so it becomes unusable after a        set time, or time or event-based auto-erase (“forgetting”)        processes    -   Utilizing hardware IDs of intelligent devices to associate        voiceprints with single device or group of devices.

While particular preferred and alternative embodiments of the presentintention have been disclosed, it will be appreciated that many variousmodifications and extensions of the above described technology may beimplemented using the teaching of this invention. All such modificationsand extensions are intended to be included within the true spirit andscope of the appended claims.

What is claimed, is:
 1. An intelligent device comprising, a processor, amemory and a clock/timer; a power source; a passcode and a voice-codestored in the memory, the voice-code indicative of the passcode; anevent manager; a user message medium; an audio input transducer;voiceprint circuitry, and evaluation circuitry; wherein the processorperforms the steps of: capturing, using the audio input transducer, auser vocalized passcode; generating, using the voiceprint circuitry, avoiceprint of the captured passcode; evaluating the voiceprint and thevoice code; and authenticating or authorizing a user.
 2. The intelligentdevice according to claim 1, wherein the voice-code is either a nativevoice-code or a synthetic voice-code.
 3. The intelligent deviceaccording to claim 1, wherein the user message medium is a visualmessage medium, an acoustic message medium, or a haptic message medium.4. The intelligent device according to claim 3, wherein the visualmessage medium is polymodal.
 5. The intelligent device according toclaim 1, further including communication circuitry.
 6. The intelligentdevice of claim 1, further including sensors and the event managerfurther performs the step of detecting an event.
 7. The intelligentdevice according to claim 6, wherein the sensor is a temperature sensor,a shock sensor, a vibration sensor, a motion sensor, a pressure sensor,a strain sensor, a chemical sensor, a radiation sensor, a humiditysensor, an acoustic sensor, or a light sensor.
 8. The intelligent deviceaccording to claim 1, further including location circuitry.
 9. Theintelligent device according to claim 8, wherein the evaluationcircuitry acts responsive to the location circuitry.
 10. The intelligentdevice according to claim 1, further including verification circuitry.11. The intelligent device according to claim 10, wherein theverification circuitry is visual verification circuitry, acousticverification circuitry or wireless verification circuitry.
 12. Theintelligent device according to claim 11, wherein the evaluationcircuitry acts responsive to the verification circuitry.
 13. Theintelligent device according to claim 1, wherein the intelligent deviceis a hardware agent.
 14. The intelligent device according to claim 13,wherein the evaluation circuitry acts responsive to stored instructionsfor each of a plurality of stakeholders, and the processor furtherperforms the step of enforcing rights for all of the stakeholdersaccording to the stored instructions.
 15. The intelligent device ofaccording to claim 1, wherein the processor effectuates actions inresponse to the evaluation step.
 16. The intelligent device according toclaim 15, wherein the actions performed by the processor include storingor presenting the results of the evaluation.
 17. The intelligent deviceaccording to claim 15, wherein the actions performed by the processorinclude generating an alarm or alert signal, communicating a message, ormodifying a message presented by a message medium of the intelligentdevice.
 18. The intelligent device according to claim 15, wherein theactions performed by the processor include generating or erasingpasscodes or voice-codes.
 19. The intelligent device according to claim15, wherein the actions performed by the processor include grantingpermissions in response to the evaluation.
 20. The intelligent deviceaccording to claim 15, wherein the actions performed by the processorinclude storing information related to the user, location or good. 21.The intelligent device of according to claim 1, further including aplurality of passcodes or a plurality of voice-codes.
 22. Theintelligent device according to claim 5, wherein the results of theevaluation step performed by the processor are transmitted.
 23. Theintelligent device of according to claim 5, wherein responsive to theevaluation step performed by the processor an alert or alarm or amessage is transmitted.
 24. An intelligent device comprising, aprocessor, a memory and a clock/timer; a power source; an event manager;a passcode and a voice-code, the voice-code related to the passcode;communication circuitry; evaluation circuitry; and wherein thecommunication circuitry receives a voiceprint, and the processorperforms the steps of evaluating the voiceprint with the storedvoice-code and authenticating or authorizing the user.
 25. Theintelligent device according to claim 24, wherein the voice-code iseither a native voice-code or a synthetic voice-code.
 26. Theintelligent device of claim 24, further including sensors.
 27. Theintelligent device according to claim 26, wherein the sensor is atemperature sensor, a shock sensor, a vibration sensor, a motion sensor,a pressure sensor, a strain sensor, a chemical sensor, a radiationsensor, a humidity sensor, an acoustic sensor, or a light sensor. 28.The intelligent device according to claim 24, further including locationcircuitry.
 29. The intelligent device according to claim 28, wherein theevaluation circuitry acts responsive to the location circuitry.
 30. Theintelligent device according to claim 24, further comprising a visualuser message medium, an acoustic message medium or a haptic user messagemedium.
 31. The intelligent device according to claim 30 wherein thevisual user message medium is polymodal.
 32. The intelligent deviceaccording to claim 30, further including verification circuitry.
 33. Theintelligent device according to claim 32, wherein the evaluationcircuitry acts responsive to the verification circuitry.
 34. Theintelligent device according to claim 24, wherein the intelligent deviceis a hardware agent.
 35. The intelligent device according to claim 34,wherein the evaluation circuitry is responsive to the storedinstructions for each of a plurality of stakeholders, and the processorfurther performs the step of enforcing rights for all of thestakeholders according to the stored instructions.
 36. The intelligentdevice of according to claim 24, wherein the processor effectuatesactions in response to the evaluation step.
 37. The intelligent deviceaccording to claim 36, wherein the actions performed by the processorinclude storing or transmitting the results of the evaluation.
 38. Theintelligent device according to claim 37, wherein the actions performedby the processor include transmitting an alarm or alert signal, or amessage.
 39. The intelligent device according to claim 36, wherein theactions performed by the processor include granting permissions inresponse to the evaluation.
 40. The intelligent device according toclaim 36, wherein the actions performed by the processor include storinginformation related to the user, location or good.
 41. The intelligentdevice according to claim 36, wherein the actions performed by theprocessor include generating or erasing passcodes or voice-codes. 42.The intelligent device of according to claim 30, wherein the processorperforms the further step of modifying or presenting a message inresponse to the evaluation.
 43. The intelligent device of according toclaim 24, further including a plurality of passcodes or a plurality ofvoice-codes.